STAIR: The STanford Artificial Intelligence Robot project

We describe an application of learning and probabilistic reasoning methods to the problem of having a robot fetch an item in response to a verbal request. Done on the STAIR (STanford AI Robot) platform, this work represents a small step towards our longer term goal of building general-purpose home assistant robots. Having a robot usefully fetch items around a home or office requires that it be able to understand a spoken command to fetch an item, that it can navigate to the location of the object (including opening doors), find and recognize the object it is asked to fetch, understand the 3d structure and position of objects in the environment, and be able to figure out how to physically pick up the object from its current location, so as to bring it back to the person making the request. By tying together different algorithms for carrying out these tasks, we recently succeeded in having the robot fetch an item in response to a verbal request. We describe below some of the key components integrated together to build this application. Probabilistic multi-resolution maps. For a robot to navigate indoor environments and open doors, it must be able to reason about about maps on the scale of 10s of meters, as well as perform manipulation that is accurate to millimeters to use door-handles. Building on the work of [2], a unified probabilistic representation is described in [6] that allows our robot to coherently and simultaneously reason using both course grid-maps of a building discretized at 10cm intervals, and very accurate models of doors that are accurate to the 1mm level. The key idea is a single representation that allows us to compute the probability of any sensor measurement (laser scan reading) given a map that has both highand low-resolution portions. This allows our robot to navigate indoors and open doors. By extending these ideas to use computer vision to recognize doors and doorhandles, we are further able to open and manipulate previously unknown doors— even ones of designs different from those in the training set—with 91% success rate. Learning to grasp unknown objects. Even though robots today can carry out tasks as complex as assembling a car, most robots are hopeless when faced with novel objects and novel environments. However, the STAIR robot must be able to grasp even previously unknown objects, if it is to be able to fetch items that did not appear in the training set (such as a stapler, coffee mug, etc. of novel shape and/or appearance). Using sensors such as stereo vision, it is extremely difficult for a robot to build an accurate 3d model of an object that it is seeing for the first time, because of occlusion, etc. However, given even a single monocular image, it is possible to obtain a rough estimate of the 3d shape of a scene. [7, 1, 4] Using multiple monocular cues, [8] describes an algorithm for finding a strategy for grasping novel objects. By further incorporating a learning algorithm for selecting grasps even in the presence of nearby obstacles, our algorithms are often able to grasp objects even in the presence of nearby clutter. Foveated vision. There are many reasons, such as context [9], that human object recognition is far superior to robotic vision. However, one reason that has been little exploited in computer object recognition is that humans use a fovea (a high resolution, central part of the retina) to obtain high resolution images of objects that they are trying to recognize. And, object recognition